Age, not autism, influences multisensory integration of speech stimuli among adults in a McGurk/MacDonald paradigm.

Differences between autistic and non-autistic individuals in perception of the temporal relationships between sights and sounds are theorized to underlie difficulties in integrating relevant sensory information. These, in turn, are thought to contribute to problems with speech perception and higher level social behaviour. However, the literature establishing this connection often involves limited sample sizes and focuses almost entirely on children. To determine whether these differences persist into adulthood, we compared 496 autistic and 373 non-autistic adults (aged 17 to 75 years). Participants completed an online version of the McGurk/MacDonald paradigm, a multisensory illusion indicative of the ability to integrate audiovisual speech stimuli. Audiovisual asynchrony was manipulated, and participants responded both to the syllable they perceived (revealing their susceptibility to the illusion) and to whether or not the audio and video were synchronized (allowing insight into temporal processing). In contrast with prior research with smaller, younger samples, we detected no evidence of impaired temporal or multisensory processing in autistic adults. Instead, we found that in both groups, multisensory integration correlated strongly with age. This contradicts prior presumptions that differences in multisensory perception persist and even increase in magnitude over the lifespan of autistic individuals. It also suggests that the compensatory role multisensory integration may play as the individual senses decline with age is intact. These findings challenge existing theories and provide an optimistic perspective on autistic development. They also underline the importance of expanding autism research to better reflect the age range of the autistic population.

Opposing serial dependencies revealed for sequences of auditory emotional stimuli.

Our percept of the world is not solely determined by what we perceive and process at a given moment in time, but also depends on what we processed recently. In the present study, we investigate whether the perceived emotion of a spoken sentence is contingent upon the emotion of an auditory stimulus on the preceding trial (i.e., serial dependence). Thereto, participants were exposed to spoken sentences that varied in emotional affect by changing the prosody that ranged from 'happy' to 'fearful'. Participants were instructed to rate the emotion. We found a positive serial dependence for emotion processing whereby the perceived emotion was biased towards the emotion on the preceding trial. When we introduced 'no-go' trials (i.e., no rating was required), we found a negative serial dependence when participants knew in advance to withhold their response on a given trial (Experiment 2) and a positive serial dependence when participants received the information to withhold their response after the stimulus presentation (Experiment 3). We therefore established a robust serial dependence for emotion processing in speech and introduce a methodology to disentangle perceptual from post-perceptual processes. This approach can be applied to the vast majority of studies investigating sequential dependencies to separate positive from negative serial dependence.

Attentional Tunneling in Pilots During a Visual Tracking Task With a Head Mounted Display.

OBJECTIVE: We examined whether active head aiming with a Helmet Mounted Display (HMD) can draw the pilot's attention away from a primary flight task. Furthermore, we examined whether visual clutter increases this effect. BACKGROUND: Head up display symbology can result in attentional tunneling, and clutter makes it difficult to identify objects. METHOD: Eighteen military pilots had to simultaneously perform an attitude control task while flying in clouds and a head aiming task in a fixed-base flight simulator. The former consisted of manual compensation for roll disturbances of the aircraft, while the latter consisted of keeping a moving visual target inside a small or large head-referenced circle. A "no head aiming" condition served as a baseline. Furthermore, all conditions were performed with or without visual clutter. RESULTS: Head aiming led to deterioration of the attitude control task performance and an increase of the amount of roll-reversal errors (RREs). This was even the case when head aiming required minimal effort. Head aiming accuracy was significantly lower when the roll disturbances in the attitude control task were large compared to when they were small. Visual clutter had no effect on both tasks. CONCLUSION: We suggest that active head aiming of HMD symbology can cause attentional tunneling, as expressed by an increased number of RREs and less accuracy on a simultaneously performed attitude control task. APPLICATION: This study improves our understanding in the perceptual and cognitive effects of (military) HMDs, and has implications for operational use and possibly (re)design of HMDs.

Perception of temporal synchrony not a prerequisite for multisensory integration.

Temporal alignment is often viewed as the most essential cue the brain can use to integrate information from across sensory modalities. However, the importance of conscious perception of synchrony to multisensory integration is a controversial topic. Conversely, the influence of cross-modal incongruence of higher level stimulus features such as phonetics on temporal processing is poorly understood. To explore the nuances of this relationship between temporal processing and multisensory integration, we presented 101 participants (ranging from 19 to 73 years of age) with stimuli designed to elicit the McGurk/MacDonald illusion (either matched or mismatched pairs of phonemes and visemes) with varying degrees of stimulus onset asynchrony between the visual and auditory streams. We asked them to indicate which syllable they perceived and whether the video and audio were synchronized on each trial. We found that participants often experienced the illusion despite not perceiving the stimuli as synchronous, and the same phonetic incongruence that produced the illusion also led to significant interference in simultaneity judgments. These findings challenge the longstanding assumption that perception of synchrony is a prerequisite to multisensory integration, support a more flexible view of multisensory integration, and suggest a complex, reciprocal relationship between temporal and multisensory processing.

Helicopter Pilot Performance and Workload in a Following Task in a Degraded Visual Environment.

BACKGROUND: In this study, we investigated the impact of a loss of horizon due to atmospheric conditions on flight performance and workload of helicopter pilots during a low-altitude, dynamic flight task in windy conditions at sea. We also examined the potential benefits of a helmet-mounted display (HMD) for this specific task.METHODS: In a fixed-based helicopter simulator, 16 military helicopter pilots were asked to follow a maneuvering go-fast vessel in a good visual environment (GVE) and in a degraded visual environment (DVE). DVE was simulated by fog, obscuring the horizon and reducing contrast. Both visual conditions were performed once with and once without an HMD, which was simulated by projecting head-slaved symbology in the outside visuals. Objective measures included flight performance, control inputs, gaze direction, and relative positioning. Subjective measures included self-ratings on performance, situation awareness, and workload.RESULTS: The results showed that in DVE the pilots perceived higher workload and were flying closer to the go-fast vessel than in GVE. Consequently, they responded with larger control inputs to maneuvers of the vessel. The availability of an HMD hardly improved flight performance but did allow the pilots to focus their attention more outside, significantly improving their situation awareness and reducing workload. These benefits were found in DVE as well as GVE conditions.DISCUSSION: DVE negatively affects workload and flight performance of helicopter pilots in a dynamic, low-altitude following task. An HMD can help improve situation awareness and lower the workload during such a task, irrespective of the visual conditions.Ledegang WD, van der Burg E, Valk PJL, Houben MMJ, Groen EL. Helicopter pilot performance and workload in a following task in a degraded visual environment. Aerosp Med Hum Perform. 2024; 95(1):16-24.

A CODE model bridging crowding in sparse and dense displays.

Visual crowding is arguably the strongest limitation imposed on extrafoveal vision, and is a relatively well-understood phenomenon. However, most investigations and theories are based on sparse displays consisting of a target and at most a handful of flanker objects. Recent findings suggest that the laws thought to govern crowding may not hold for densely cluttered displays, and that grouping and nearest neighbour effects may be more important. Here we present a computational model that accounts for crowding effects in both sparse and dense displays. The model is an adaptation and extension of an earlier model that has previously successfully accounted for spatial clustering, numerosity and object-based attention phenomena. Our model combines grouping by proximity and similarity with a nearest neighbour rule, and defines crowding as the extent to which target and flankers fail to segment. We show that when the model is optimized for explaining crowding phenomena in classic, sparse displays, it also does a good job in capturing novel crowding patterns in dense displays, in both existing and new data sets. The model thus ties together different principles governing crowding, specifically Bouma's law, grouping, and nearest neighbour similarity effects.

Visual crowding: Double dissociation between orientation and brightness judgments.

Peripherally presented objects are often more difficult to identify when located in cluttered visual environments than when presented in isolation, a phenomenon known as visual crowding. Crowding tends to be stronger when target and nearby flanking elements are composed of similar sets of features. This study investigates the extent to which target-flanker orientation and/or color similarity determines luminance and orientation performance across different tasks under identical stimulus conditions. Targets were near-vertical Gabor patches defined by modulating only the green component of the RGB display. Subjects performed both target luminance and orientation discrimination tasks in separate blocks while both flanker hue (green or red flankers) and orientation (vertical or horizontal flankers) were manipulated as a function of target-flanker separation. We find strong evidence for a double dissociation between task and the specific set of features by which target-flanker similarity is defined. Whereas luminance judgments were highly contingent upon target-flanker hue similarity, orientation judgments showed the inverse pattern, largely contingent upon flanker orientation. The magnitude of this double dissociation decreased with target-flanker separation, at a rate predicted by Bouma's law. This specific pattern of performance provides strong support for the idea that crowding operates independently for the most part within orientation and color domains. That luminance judgments are constrained by target-flanker flanker hue similarity and, to a far lesser extent, by target-flanker orientation similarity suggests that the neural mechanisms responsible for mediating perceived luminance are principally linked to those mediating stimulus hue independent of those mediating stimulus orientation.

No differences between adults with and without autism in audiovisual synchrony perception.

LAY ABSTRACT: It has been known for a long time that individuals diagnosed with autism spectrum disorder perceive the world differently. In this study, we investigated how people with or without autism perceive visual and auditory information. We know that an auditory and a visual stimulus do not have to be perfectly synchronous for us to perceive them as synchronous: first, when the two are within a certain time window (temporal binding window), the brain will tell us that they are synchronous. Second, the brain can also adapt quickly to audiovisual asynchronies (rapid recalibration). Although previous studies have shown that people with autism spectrum disorder have different temporal binding windows, and less rapid recalibration, we did not find these differences in our study. However, we did find that both processes develop over age, and since previous studies tested only young people (children, adolescents, and young adults), and we tested adults from 18 to 55 years, this might explain the different findings. In the end, there might be quite a complex story, where people with and without autism spectrum disorder perceive the world differently, even dependent on how old they are.

Electrodermal activity as an index of food neophobia outside the lab.

Introduction: Understanding how food neophobia affects food experience may help to shift toward sustainable diets. Previous research suggests that individuals with higher food neophobia are more aroused and attentive when observing food-related stimuli. The present study examined whether electrodermal activity (EDA), as index of arousal, relates to food neophobia outside the lab when exposed to a single piece of food. Methods: The EDA of 153 participants was analyzed as part of a larger experiment conducted at a festival. Participants completed the 10-item Food Neophobia Scale. Subsequently, they saw three lids covering three foods: a hotdog labeled as "meat", a hotdog labeled as "100% plant-based", and tofu labeled as "100% plant-based". Participants lifted the lids consecutively and the area-under-the-curve (AUC) of the skin conductance response (SCR) was captured between 20 s before and 20 s after each food reveal. Results: We found a significant positive correlation between food neophobia and AUC of SCR during presentation of the first and second hotdog and a trend for tofu. These correlations remained significant even when only including the SCR data prior to the food reveal (i.e., an anticipatory response). Discussion: The association between food neophobia and EDA indicates that food neophobic individuals are more aroused upon the presentation of food. We show for the first time that the anticipation of being presented with food already increased arousal for food neophobic individuals. These findings also indicate that EDA can be meaningfully determined using wearables outside the lab, in a relatively uncontrolled setting for single-trial analysis.

Linking Categorical and Dimensional Approaches to Assess Food-Related Emotions.

Reflecting the two main prevailing and opposing views on the nature of emotions, emotional responses to food and beverages are typically measured using either (a) a categorical (lexicon-based) approach where users select or rate the terms that best express their food-related feelings or (b) a dimensional approach where they rate perceived food items along the dimensions of valence and arousal. Relating these two approaches is problematic since a response in terms of valence and arousal is not easily expressed in terms of emotions (like happy or disgusted). In this study, we linked the dimensional approach to a categorical approach by establishing mapping between a set of 25 emotion terms (EsSense25) and the valence-arousal space (via the EmojiGrid graphical response tool), using a set of 20 food images. In two 'matching' tasks, the participants first imagined how the food shown in a given image would make them feel and then reported either the emotional terms or the combination of valence and arousal that best described their feelings. In two labeling tasks, the participants first imagined experiencing a given emotion term and then they selected either the foods (images) that appeared capable to elicit that feeling or reported the combination of valence and arousal that best reflected that feeling. By combining (1) the mapping between the emotion terms and the food images with (2) the mapping of the food images to the valence-arousal space, we established (3) an indirect (via the images) mapping of the emotion terms to the valence-arousal space. The results show that the mapping between terms and images was reliable and that the linkages have straightforward and meaningful interpretations. The valence and arousal values that were assigned to the emotion terms through indirect mapping to the valence-arousal space were typically less extreme than those that were assigned through direct mapping.

Serial Dependence of Emotion Within and Between Stimulus Sensory Modalities.

How we perceive the world is not solely determined by what we sense at a given moment in time, but also by what we processed recently. Here we investigated whether such serial dependencies for emotional stimuli transfer from one modality to another. Participants were presented a random sequence of emotional sounds and images and instructed to rate the valence and arousal of each stimulus (Experiment 1). For both ratings, we conducted an intertrial analysis, based on whether the rating on the previous trial was low or high. We found a positive serial dependence for valence and arousal regardless of the stimulus modality on two consecutive trials. In Experiment 2, we examined whether passively perceiving a stimulus is sufficient to induce a serial dependence. In Experiment 2, participants were instructed to rate the stimuli only on active trials and not on passive trials. The participants were informed that the active and passive trials were presented in alternating order, so that they were able to prepare for the task. We conducted an intertrial analysis on active trials, based on whether the rating on the previous passive trial (determined in Experiment 1) was low or high. For both ratings, we again observed positive serial dependencies regardless of the stimulus modality. We conclude that the emotional experience triggered by one stimulus affects the emotional experience for a subsequent stimulus regardless of their sensory modalities, that this occurs in a bottom-up fashion, and that this can be explained by residual activation in the emotional network in the brain.

Helmholtz Versus Haute Couture: How Horizontal Stripes and Dark Clothes Make You Look Thinner.

In Helmholtz's illusion, a square with horizontal stripes appears taller than an identical square with vertical stripes. This effect has also been observed in experiments with human stimuli, where a human figure wearing a dress with horizontal stripes appears thinner than a drawing clad in vertical stripes. These findings do not agree with the common belief that clothes with horizontal stripes make someone appear wider, neither do they disentangle whether the horizontal or vertical stripes account for the thinning effect. In the present study, we focused on the effect of horizontal stripes in clothes comparing horizontal stripes against no-stripes (not against vertical; Experiments 1 and 2), using photos of a real-life female model, and controlling for the average luminance of the stripes (Experiment 2). Results showed that horizontal stripes and lower luminance have-independently-a small-to-moderate thinning effect on the perceived size of the body, and the effect is larger when the two variables are combined. In Experiment 3, we further show that the thinning effect due to the luminance of the dress is enhanced when the general background gets darker.

Shrinking Bouma's window: How to model crowding in dense displays.

In crowding, perception of a target deteriorates in the presence of nearby flankers. Traditionally, it is thought that visual crowding obeys Bouma's law, i.e., all elements within a certain distance interfere with the target, and that adding more elements always leads to stronger crowding. Crowding is predominantly studied using sparse displays (a target surrounded by a few flankers). However, many studies have shown that this approach leads to wrong conclusions about human vision. Van der Burg and colleagues proposed a paradigm to measure crowding in dense displays using genetic algorithms. Displays were selected and combined over several generations to maximize human performance. In contrast to Bouma's law, only the target's nearest neighbours affected performance. Here, we tested various models to explain these results. We used the same genetic algorithm, but instead of selecting displays based on human performance we selected displays based on the model's outputs. We found that all models based on the traditional feedforward pooling framework of vision were unable to reproduce human behaviour. In contrast, all models involving a dedicated grouping stage explained the results successfully. We show how traditional models can be improved by adding a grouping stage.

Where and when to look: Sequential effects at the millisecond level.

Learning and imitating a complex motor action requires to visually follow complex movements, but conscious perception seems too slow for such tasks. Recent findings suggest that visual perception has a higher temporal resolution at an unconscious than at a conscious level. Here we investigate whether high-temporal resolution in visual perception relies on prediction mechanisms and attention shifts based on recently experienced sequences of visual information. To that aim we explore sequential effects during four different simultaneity/asynchrony discrimination tasks. Two stimuli are displayed on each trial with varying stimulus onset asynchronies (SOA). Subjects decide whether the stimuli are simultaneous or asynchronous and give manual responses. The main finding is an advantage for different-order over same-order trials, when subjects decided that stimuli had been simultaneous on Trial t - 1 , and when Trial t is with an SOA slightly larger than Trial t - 1, or equivalent. The advantage for different-order trials disappears when the stimuli change eccentricity but not direction between trials (Experiment 2), and persists with stimuli displayed in the centre and unlikely to elicit a sense of direction (Experiment 4). It is still observed when asynchronies on Trial t - 1 are small and undetected (Experiment 3). The findings can be explained by an attention shift that is precisely planned in time and space and that incidentally allows subjects to detect an isolated stimulus on the screen, thus helping them to detect an asynchrony.

Learned prioritization yields attentional biases through selection history.

While numerous studies have provided evidence for selection history as a robust influence on attentional allocation, it is unclear precisely which behavioral factors can result in this form of attentional bias. In the current study, we focus on "learned prioritization" as an underlying mechanism of selection history and its effects on selective attention. We conducted two experiments, each starting with a training phase to ensure that participants learned different stimulus priorities. This was accomplished via a visual search task in which a specific color was consistently more relevant when presented together with another given color. In Experiment 1, one color was always prioritized over another color and inferior to a third color, such that each color had an equal overall priority by the end of the training session. In Experiment 2, the three different colors had unequal priorities at the end of the training session. A subsequent testing phase in which participants had to search for a shape-defined target showed that only stimuli with unequal overall priorities (Experiment 2) affected attentional selection, with increased reaction times when a distractor was presented in a previously high-priority compared with a low-priority color. These results demonstrate that adopting an attentional set where certain stimuli are prioritized over others can result in a lingering attentional bias and further suggest that selection history does not equally operate on all previously selected stimuli. Finally, we propose that findings in value-driven attention studies where high-value and low-value signaling stimuli differentially capture attention may be a result of learned prioritization rather than reward.

A Thickness Illusion: Horizontal Is Perceived as Thicker than Vertical.

We report two psychophysical experiments that investigate a visual illusion that is considered common knowledge among type designers, but has never been studied scientifically. Specifically, the thickness of a horizontal line is overestimated in relation to that of a vertical line. Experiment 1 confirmed the existence of the illusion. In Experiment 2, we replicated the effect and showed that the illusion is closely related to the vertical-horizontal illusion, in which the length of a vertical line is overestimated in comparison to a horizontal one. Both the overestimation of thickness and length is larger when the stimulus is surrounded by a horizontally elongated frame, as opposed to a vertically elongated frame. We discuss potential explanations for the thickness illusion and its relation to the vertical-horizontal illusion.

Positive sequential dependency for face attractiveness perception.

Recent findings from several groups have demonstrated that visual perception at a given moment can be biased toward what was recently seen. This is true both for basic visual attributes and for more complex representations, such as face identity, gender, or expression. This assimilation to the recent past is a positive serial dependency, similar to a temporal averaging process that capitalizes on short-term correlations in visual input to reduce noise and boost perceptual continuity. Here we examine serial dependencies in face perception using a simple attractiveness rating task and a rapid series of briefly presented face stimuli. In a series of three experiments, our results confirm a previous report that face attractiveness exhibits a positive serial dependency. This intertrial effect is not only determined by face attractiveness on the previous trial, but also depends on the faces shown up to five trials back. We examine the effect of stimulus presentation duration and find that stimuli as brief as 56 ms produce a significant positive dependency similar in magnitude to that produced by stimuli presented for 1,000 ms. We observed stronger positive dependencies between same-gender faces, and found a task dependency: Alternating gender discrimination trials with attractiveness rating trials produced no serial dependency. In sum, these findings show that a perception-stabilizing assimilation effect operates in face attractiveness perception that is task dependent and is acquired surprisingly quickly.

Changes (but not differences) in motion direction fail to capture attention.

In this study we investigated under what conditions motion direction changes pop out in continuously moving target/distractor environments. Participants were presented with vertically oriented Gabor patches whose carrier components drifted at a constant speed from left to right and then reversed direction. On any given trial, one of these elements was nominated as the target and the remaining elements were distractors. Distractor elements all changed direction simultaneously. The distractors either moved in a homogeneous manner (i.e. all moved in the same direction), or in a heterogeneous manner (i.e. direction was randomized). The target moved with a similar spatio-temporal trajectory as the distractors from left to right (or vice versa), but changed direction asynchronously with respect to the distracting elements. The participants' task was to locate this deviant (target) Gabor patch. We show that a motion direction change pops out (as indicated by the absence of a set size effect) when the surrounding distractors move in a homogeneous direction. When the distractors moved in heterogenous directions, a similar pop out effect was observed when the set size was small (≤5 elements), but not when it was large. This suggests that motion direction changes capture attention only when the change results in a unique direction of motion. Consistent with this finding we also show that a moving target (without direction change) captures attention in cases in which all distractors recently changed direction. This corroborates the idea that, in addition to direction cues, the temporal uniqueness of a change in an object's direction (or absence, thereof) relative to surrounding objects is a cue capable of capturing attention.

Attentional guidance varies with display density.

The aim of the present study was to investigate how display density affects attentional guidance in heterogeneous search displays. In Experiment 1 we presented observers with heterogeneous sparse and dense search displays which were adaptively changed over the course of the experiment using genetic algorithms. We generated random displays, and based upon fastest search times, the displays that allowed most efficient search were selected to generate new displays for the next generations, thus revealing which properties facilitated or inhibited target search across display densities. The results showed that the prevalence of distractors sharing the target color was substantially reduced over generations in sparse displays. Dense displays also evolved to contain less distractors sharing the target color but only when the orientation of the distractors resembled the target orientation. More importantly, spatial analyses revealed that changes across generations occurred across all areas in sparse displays but were confined to occur around the target location only in dense displays. In Experiment 2, in which we used a factorial design, we showed that the presence of potentially interfering distractors in the target area affected search in dense displays but not in sparse displays. Together the results suggest that the role of salience-driven attentional guidance is larger in dense than sparse displays even in the absence of display homogeneity.

Rapid Audiovisual Temporal Recalibration Generalises Across Spatial Location.

Recent exposure to asynchronous multisensory signals has been shown to shift perceived timing between the sensory modalities, a phenomenon known as 'temporal recalibration'. Recently, Van der Burg et al. (2013, J Neurosci, 33, pp. 14633-14637) reported results showing that recalibration to asynchronous audiovisual events can happen extremely rapidly. In an extended series of variously asynchronous trials, simultaneity judgements were analysed based on the modality order in the preceding trial and showed that shifts in the point of subjective synchrony occurred almost instantaneously, shifting from one trial to the next. Here we replicate the finding that shifts in perceived timing occur following exposure to a single, asynchronous audiovisual stimulus and by manipulating the spatial location of the audiovisual events we demonstrate that recalibration occurs even when the adapting stimulus is presented in a different location. Timing shifts were also observed when the adapting audiovisual pair were defined only by temporal proximity, with the auditory component presented over headphones rather than being collocated with the visual stimulus. Combined with previous findings showing that timing shifts are independent of stimulus features such as colour and pitch, our finding that recalibration is not spatially specific provides strong evidence for a rapid recalibration process that is solely dependent on recent temporal information, regardless of feature or location. These rapid and automatic shifts in perceived synchrony may allow our sensory systems to flexibly adjust to the variation in timing of neural signals occurring as a result of delayed environmental transmission and differing neural latencies for processing vision and audition.